Skip to content

Add PHPCS linting for PHP blocks in feature files - #340

Merged
swissspidy merged 22 commits into
mainfrom
try/phpcs-in-feature-files
Aug 26, 2026
Merged

Add PHPCS linting for PHP blocks in feature files#340
swissspidy merged 22 commits into
mainfrom
try/phpcs-in-feature-files

Conversation

@swissspidy

@swissspidy swissspidy commented Jul 23, 2026

Copy link
Copy Markdown
Member

Not sure yet if really worth it, will need some testing.

Summary by CodeRabbit

  • New Features

    • Added validation and automatic formatting for PHP code embedded in feature files.
    • Added tooling to extract embedded PHP for checks and synchronize formatting fixes back into feature files.
  • Bug Fixes

    • Improved temporary-file cleanup, including when checks are interrupted.
    • Preserved and reported code-quality check failures while allowing applicable checks to complete.
  • Style

    • Standardized formatting in feature-file test scenarios.
  • Tests

    • Added comprehensive coverage for PHP extraction, validation, formatting, and synchronization.

@swissspidy swissspidy added the scope:testing Related to testing label Jul 23, 2026
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

Next included review available in 2 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e8cac7d1-397e-47e5-96dd-a58641f551a7

📥 Commits

Reviewing files that changed from the base of the PR and between 2c962f3 and 77737d8.

📒 Files selected for processing (11)
  • .readme-partials/USING.md
  • bin/run-phpcbf-cleanup
  • bin/run-phpcs-tests
  • features/behat-steps.feature
  • phpcs/feature-files.sh
  • tests/tests/FeatureFilesTestCase.php
  • tests/tests/TestExtractFeaturePhp.php
  • tests/tests/TestPhpStanFeatureFiles.php
  • utils/extract-feature-php.php
  • utils/feature-php-blocks.php
  • utils/phpstan-feature-files.php
📝 Walkthrough

Walkthrough

The change adds PHP extraction and synchronization for .feature files. PHPCS and PHPCBF now validate or format extracted blocks, preserve accumulated failures, and clean temporary files. PHPUnit tests cover extraction, updates, validation, and error handling.

Changes

Feature PHP checks

Layer / File(s) Summary
Feature PHP extraction and synchronization
utils/extract-feature-php.php
Adds a CLI utility that validates targets, extracts PHP blocks into padded temporary files, and synchronizes processed content back into .feature files.
Extraction utility validation
tests/tests/TestExtractFeaturePhp.php
Adds tests for extraction, target validation, generated-file cleanup, indentation, PHP tags, padding, synchronization, and failure cases.
PHPCS and PHPCBF script integration
bin/run-phpcbf-cleanup, bin/run-phpcs-tests, features/*.feature
Accumulates command statuses, conditionally processes PHP blocks in feature files, cleans temporary files, and applies minor syntax-formatting changes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Developer
  participant run_phpcs_tests
  participant extract_feature_php
  participant PHPCS
  participant FeatureFiles
  Developer->>run_phpcs_tests: run checks
  run_phpcs_tests->>extract_feature_php: extract PHP blocks
  extract_feature_php->>FeatureFiles: read .feature files
  extract_feature_php-->>run_phpcs_tests: create temporary PHP files
  run_phpcs_tests->>PHPCS: check temporary PHP files
  PHPCS-->>run_phpcs_tests: return check status
  run_phpcs_tests-->>Developer: exit with accumulated status
Loading

Suggested reviewers: brianhenryie

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 36.36% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding PHPCS linting for PHP blocks in feature files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch try/phpcs-in-feature-files

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@swissspidy
swissspidy marked this pull request as ready for review August 4, 2026 08:42
@swissspidy
swissspidy requested a review from a team as a code owner August 4, 2026 08:42
Copilot AI lite review requested due to automatic review settings August 4, 2026 08:42

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

Fixes for the PHP block extraction and synchronization:

* Place an added `<?php` tag on the docstring delimiter line, so that it
  is no longer overwritten by the first line of code and blocks without an
  opening tag are actually linted as PHP.
* Only treat a docstring as PHP when it follows a step creating a PHP file
  or starts with `<?php`. Expectations such as `Then the wp-config.php
  file should contain:` are not necessarily valid PHP and must not be
  reformatted.
* Determine the indentation to restore from the first line holding code,
  as a leading empty line would otherwise be used as indentation.
* Remove only the padding added during extraction instead of all leading
  and trailing empty lines, preserving intentional ones.
* Keep empty lines leading up to an opening tag, so that reported line
  numbers keep matching.
* Only remove files from the target directory that a previous extraction
  created, and refuse target directories that hold the feature files
  themselves.
* Only treat the first argument as an action when it is one, so that
  passing two directories does not swap source and target.
* Report failures on STDERR and through the exit code.
* Skip a block instead of writing it back when the extracted file no
  longer lines up with the feature file.

Fixes for the scripts running the checks:

* Propagate failures of the extraction, PHPCS, PHPCBF and synchronization
  steps, and skip synchronization when extraction failed.
* Preserve the PHPCS exit status by writing the report to a file instead
  of piping it through `sed`.
* Keep nested directories when normalizing reported feature file paths.

Adds unit tests covering extraction and synchronization.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KVnFXuhCGs4NT2A7FmDo4e
coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

@swissspidy

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Extraction took a number of leading whitespace characters off every line
of a block and syncing put the indentation of the block's first line back,
which is only the same thing when that line is the least indented one. A
block whose opening tag sits deeper than the code below it drifted further
to the right on every `composer phpcbf`, and one mixing tabs and spaces
came back with one swapped for the other, both without a single sniff
having fired. Extraction now takes off the indentation prefix that all
lines of a block share, and syncing puts exactly that prefix back.

Extraction also counted a docstring opening with `<?php` as a PHP block
while syncing only accepted one belonging to a step creating a `.php`
file. A docstring stating an expectation about the contents of a file was
therefore checked, fixed, and then refused, which failed the run and threw
the fixes away. Only the step decides now: reformatting an expectation
would make it stop matching what it is checked against, so the analysis in
`phpstan-feature-files.php` keeps the second rule and the fixer does not.

Both scripts let the filesystem root through as an extraction target. The
check meant to catch it appends a separator to the target before comparing,
which never matches a root that already ends in one, so `remove_extracted_files()`
would have walked the whole filesystem removing every empty directory it
could. Roots are now rejected up front, and the walk refuses to start on
one even if a caller gets past that.

Also names the feature file and the line in the two messages reporting a
block that could not be synced, so it is possible to tell which one it was.
The root of this package was derived from `dirname "$0"`, but Composer
installs these scripts as a symlink in the vendor binary directory, where
that resolves to `vendor` instead. `vendor/utils/extract-feature-php.php`
does not exist, so the check was skipped in every package using the testing
framework and only ever ran here, where `bin` and `utils` are siblings.
The symlink is now resolved first, the way `run-phpstan-tests` already does.

PHPCS truncates a reported path from the left once it grows past the width
of the report, which happened before the temporary directory was rewritten
out of it and left findings pointing at `...N1dTe6Y/some/feature.feature`.
Passing `--basepath` reduces the paths to the part worth showing, so the
report needs a single anchored substitution and no longer depends on the
length of a path it does not control.

The list of sniffs that do not apply to a block was spelled out in both
scripts. A future edit to one of them would have had the fixer rewrite
feature files over something the check never reports, so the list moves to
`phpcs/feature-files.sh`, which both read. That also gets the fixer the
`--warning-severity=0` only the check was passing. A package replaces the
defaults by adding a `phpcs-feature-files.xml` ruleset to its root.

The blocks are left alone when a run is narrowed down to a path, as in
`composer phpcs -- src/`, since such an argument is about the files of the
package itself rather than about its feature files.
Adds a section alongside the one for the static analysis, covering which
docstrings are checked and why a docstring merely opening with `<?php` is
not among them, where the defaults live, and how a package replaces them.

The README is regenerated from the partials on push to main.
`extract-feature-php.php` and `phpstan-feature-files.php` each carried
their own copy of the checks deciding which docstrings hold a PHP block and
where a block may be extracted to, in the same namespace and byte for byte
the same. Nothing catches such a pair drifting apart: the two run as
separate processes, `utils` is not autoloaded, and the static analysis only
looks at `src` and `tests`. The copies had already started to diverge, with
a fix for an extraction target resolving to the filesystem root landing in
one of them first.

They move to `utils/feature-php-blocks.php`, which both scripts pull in
themselves. It stays out of the autoloader, as loading a set of functions
into every package using the framework buys nothing.

`remove_extracted_files()` takes the pattern matching the extracted file
names as an argument, since the two tools name their files differently, and
the manifest that only the analysis writes is removed by the analysis.
The loop walking a feature file for docstrings existed twice: as
`collect_blocks()` in the analysis and inlined in the extractor of the code
style check, which is where the two had already come to disagree about what
counts as a PHP block. The check needs the narrower rule, since it writes a
block back and reformatting an expectation about the contents of a file
would make it stop matching, while the analysis wants every block it can
read.

Rather than a flag deciding that inside the shared loop, a block now comes
back saying which rule recognised it and whether it brings its own opening
tag, and each tool takes it from there. That also drops the separate pass
the check made over a block to answer the second question.

The walk over the feature files of a directory is shared along with it. The
code style check gains the stable order the analysis already relied on.
Both tools took the indentation a block shares off its lines before handing
them to PHP_CodeSniffer or PHPStan, and each did it its own way. The
analysis removed a number of characters, namely the smallest number of
leading whitespace characters any line of the block carried, which is only
the same thing as removing the shared prefix while every line is indented
with the same characters. It now uses the same `get_common_indent()` the
code style check does.

For a well-formed docstring the two agree, as every line of a block then
starts with the indentation of the docstring, and the analysis of every
feature file in config-command, entity-command and scaffold-command comes
out byte for byte the same. They part ways over a line indented with a tab
where the rest of the block uses spaces: the old computation would take the
tab off one line and a single space off another, leaving the block with an
indentation it never had. Nothing is written back on this side, so unlike
the same defect on the code style side this was not corrupting anything,
but it is one implementation now rather than two, and the one that is left
cannot lose the distinction.
@swissspidy

Copy link
Copy Markdown
Member Author

@coderabbitai resume

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown
✅ Action performed

Reviews resumed.

Both scripts are exercised the way they are used, by running them over a
directory of feature files written into a temporary directory, so both test
classes carried the same setup: creating and removing that directory,
writing a feature file into it, running the script from within it, and
reading back what it wrote.

The scaffolding moves to `FeatureFilesTestCase`, which the two classes
extend. Each names the script it is about and the flags to run it with, and
the rest follows: the temporary directory is still named after the script,
and the code style check still runs it without loading `php.ini` while the
analysis needs one loaded for ext-tokenizer.

The listing of what extraction wrote now leaves out files that are not PHP
in both, rather than only in the one whose target directory holds a
manifest. Extraction only ever writes `.php` files, so anything else in
there came from somewhere else.
@swissspidy swissspidy added this to the 5.2.4 milestone Aug 26, 2026
@swissspidy
swissspidy merged commit 913cd94 into main Aug 26, 2026
65 checks passed
@swissspidy
swissspidy deleted the try/phpcs-in-feature-files branch August 26, 2026 17:37
swissspidy added a commit that referenced this pull request Aug 26, 2026
Conflict in bin/run-phpcs-tests: #340 restructured it into a standard
run plus a run over the PHP blocks in feature files, the same shape the
PHPStan runner already has.

Resolved by taking that structure and re-applying WP_CLI_TEST_QUIET to
the standard run only, which is where this branch had it and which
matches how the PHPStan runner treats its own second section. The block
check keeps the default report on purpose: its findings are rewritten
back onto the feature files with a sed over the "FILE:" headers, which
a compact report would not produce.
swissspidy added a commit that referenced this pull request Aug 26, 2026
Brings in the README regeneration from #361, which adds the section
#340 had only written into .readme-partials/USING.md. It merged
cleanly with this branch's own README additions.

Checked afterwards that every section the partial defines still matches
its generated counterpart byte for byte, so the next regeneration stays
a no-op.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope:testing Related to testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants